Activity-Dependent Generation of Simulated Neural Circuits

ABSTRACT

A simulated neural circuit includes a plurality of simulated neurons. The simulated neurons have input branches that are configured to connect to a plurality of inputs and activate in response to activity in the inputs to which they are connected. In addition, the simulated neurons are configured to activate in response to activity in their input branches. Initial connections are formed between various input branches and various inputs and a set of the inputs are activated. Thereafter, the stability of connections between input branches and inputs to which they are connected is moderated based on the activated set of inputs and a pattern of activity generated in the input branches and simulated neurons in response to the activated set of inputs.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority from U.S. Provisional PatentApplication Ser. No. 60/895,334, filed on Mar. 16, 2007 and entitled“Activity Dependent Growth of Synthetic Neural Circuits” and from U.S.Provisional Patent Application Ser. No. 60/895,627, filed on Mar. 19,2007 and entitled “Activity Dependent Growth of Synthetic NeuralCircuits,” both of which are incorporated herein by reference in theirentirety.

TECHNICAL FIELD

This disclosure relates to generating simulated neural circuits.

BACKGROUND

Attempts have been made to create simulated neural circuits that includeproperties of biological neural circuits.

Biological neural circuits are made up of an incredibly dense meshworkof numerous, complex, tree-like units called “neurons.” Each neuronmakes at least thousands of connections with other neurons. It is atthese connections or “synapses” that information is transferred betweenthe neurons. The pattern of these connections, that is the wiring of theneural circuit, in large part defines the circuit's function. Thesensory performance of biological organisms makes it clear that suchcircuitry is capable of very sophisticated and powerful processing.However, unlike conventional electronic circuits, this extraordinarymeshwork wires itself. That is to say, the pattern of synapticconnections in the neural circuit is not determined a priori with anexplicitly encoded point-to-point wiring diagram, but instead is formedthrough mechanisms that allow self-organization of the circuit as partof the functioning of the neurons. The pattern of wiring embedsinformation imparted both genetically as well as that learned byexposure to the sensory environment. This process of wiring may continuethroughout the life of the organism. All real neural circuits wirethemselves. Therefore, in seeking to create simulated neural circuitswhich emulate some of the function of these biological circuits, it maybe advantageous to generate simulated circuits that also wirethemselves. After such simulated circuits have been generated, theysubsequently may be implemented and fabricated, for example, asintegrated circuits, for real-life applications.

SUMMARY

A simulated neural circuit includes neural elements. Like biologicalneurons that include dendritic trees as inputs and axons as outputs,each neural element may include inputs in the form of simulateddendritic trees and outputs in the form of simulated axons. Thesimulated dendritic trees of some neural elements and the simulatedaxons of other neural elements are wired together as the simulatedneural circuit is generated.

During generation of the neural circuit, different inputs to the neuralcircuit may be activated at different times, and these activated inputsmay result in activity in branches of the dendritic trees as well as incertain ones of the axons. This activity is used to influence thegeneration and wiring of the simulated neural circuit. For example, aconnection between a simulated dendritic branch and a simulated axon maybe formed and/or stabilized when activation of the simulated axon isclosely followed by activation of the dendritic branch such that thereis a strong correlation between the activity in the simulated axon andthe activation or firing of the simulated dendritic branch.

At other times during the generation of the simulated neural circuit, aconnection between a simulated dendritic branch and a simulated axon maybe destabilized or removed when activation of the simulated axon is notclosely followed by activation of the dendritic branch, or whenactivation of the dendritic branch occurs at a time when the simulatedaxon has not been recently activated, such that there is a relativelyweak correlation between the activation of the simulated axon andactivation of the dendritic branch.

In addition to having activity in the simulated neural circuit stabilizeor destabilize connections between different neural elements, theactivity also may be used to control how the axons and the dendriticbranches are formed. For example, a simulated axon may be influenced togrow toward a simulated dendritic branch that frequently is active.

In a simulated neural circuit, the input branches of a neural element(i.e., the dendritic branches of the neural element) may be connected tothe inputs of the simulated neural circuit and/or to the outputs (i.e.,the axons) of other neural elements.

Particular aspects of the general concepts of activity dependentgeneration of simulated neural circuits are described below.

In one aspect, a simulated neural circuit includes simulated neurons,each of which has input branches that are configured to connect toinputs and activate in response to activity in the inputs to which theyare connected. In addition, the simulated neurons are configured toactivate in response to activity in their input branches. Initialconnections are formed between various input branches and various inputsand a set of the inputs are activated. Thereafter, the stability ofconnections between input branches and inputs to which they areconnected is moderated based on the activated set of inputs and apattern of activity generated in the input branches and simulatedneurons in response to the activated set of inputs.

Implementations may include one or more of the following features. Forexample, the connections between input branches and inputs to which theyare connected may be weakened, strengthened, formed, and severingconnections between input branches and inputs based on the activated setof inputs and a pattern of activity generated in the input branches andsimulated neurons in response to the activated set of inputs.

A connection between a particular input branch and a particular inputmay be weakened when the particular input was not among the activatedset of inputs, the particular input branch remained inactive in responseto the activated set of inputs, and the simulated neuron to which theparticular input branch belongs was activated in response to theactivated set of inputs. Likewise, a connection between a particularinput branch and a particular input may be weakened when the particularinput was not among the activated set of inputs, the particular inputbranch was activated in response to the activated set of inputs, and thesimulated neuron to which the particular input branch belongs remainedinactive in response to the activated set of inputs. Additionally oralternatively, a connection between a particular input branch and aparticular input may be weakened when the particular input was not amongthe activated set of inputs, the particular input branch was activatedin response to the activated set of inputs, and the simulated neuron towhich the particular input branch belongs was activated in response tothe activated set of inputs.

Similarly, a connection between a particular input branch and aparticular input may be weakened when the particular input was among theactivated set of inputs, the particular input branch remained inactivein response to the activated set of inputs, and the simulated neuron towhich the particular input branch belongs remained inactive in responseto the activated set of inputs. Furthermore, a connection between aparticular input branch and a particular input may be weakened when theparticular input was among the activated set of inputs, the particularinput branch was activated in response to the activated set of inputs,and the simulated neuron to which the particular input branch belongsremained inactive in response to the activated set of inputs.

The connections between input branches and inputs to which they areconnected also may be strengthened based on the activated set of inputsand a pattern of activity generated in the input branches and simulatedneurons in response to the activated set of inputs. For example, aconnection between a particular input branch and a particular input maybe strengthened when the particular input was among the activated set ofinputs, the particular input branch was activated in response to theactivated set of inputs, and the simulated neuron to which theparticular input branch belongs remained inactive in response to theactivated set of inputs. Additionally or alternatively, a connectionbetween a particular input branch and a particular input may bestrengthened when the particular input was among the activated set ofinputs, the particular input branch was activated in response to theactivated set of inputs, and the simulated neuron to which theparticular input branch belongs was activated in response to theactivated set of inputs.

In some implementations, the initial connections between various inputbranches and various inputs may be formed based on the activated set ofinputs. For example, initial connections between various input branchesand various inputs may be formed based on a pattern of activitygenerated in the input branches and the simulated neurons in response tothe activated set of inputs.

In another aspect, a first array of simulated neurons having outputbranches is generated and a second array of simulated neurons havinginput branches is generated. Thereafter, a set of the output branches ofthe first array of simulated neurons is activated, and one or more ofthe input branches of the simulated neurons of the second array ofsimulated neurons are grown such that the growth of at least oneparticular input branch is influenced in the direction of one of theactivated output branches of the first array of simulated neurons.

In some implementations, the release of an attractant may be simulatedby one or more of the activated output branches in response to theactivation of the set of output branches and the particular input branchmay be configured to grow in the direction of the attractant. Therefore,after the release of the attractant, the particular branch may grow inthe direction of the attractant.

The various aspects, implementations, and features may be implementedusing, for example, one or more of a method, an apparatus, a system,tool, or processing device for performing a method, a program or otherset of instructions, an apparatus that includes a program or a set ofinstructions, and a computer program embodied in a tangible computerreadable medium. The tangible computer readable medium may include, forexample, instructions, software, images, and other data.

The details of one or more implementations are set forth in theaccompanying drawings and the description below. Other features will beapparent from the description and the drawings, and the claims.

DESCRIPTION OF DRAWINGS

FIGS. 1 a-1 h are block diagrams of a simulated neuron.

FIGS. 2 a-2 c are screenshots of a graphical user interface forgenerating simulated neurons.

FIG. 3 is a flowchart of a process for generating a simulated neuron.

FIGS. 4 a and 4 b are block diagrams of an array of simulated neurons.

FIGS. 5 a and 5 b are screenshots of a graphical user interface of asystem for generating simulated neural arrays.

FIG. 6 is a flowchart of a process for generating a simulated neuralarray.

FIG. 7 a is a block diagram of an output array of simulated neuronsconnected to an input array of simulated neurons.

FIGS. 7 b-7 d are block diagrams of a simulated neural circuit formed bygenerating and connecting the output axons of the output array ofsimulated neurons and the input dendrites of the input array ofsimulated neurons of FIG. 7 a.

FIGS. 8 a-8 c are screenshots of a graphical user interface of a systemfor generating simulated neural circuits.

FIGS. 9 a-9 c are screenshots of the graphical user interface of FIG. 8that illustrate the propagation of signals through simulated neuralcircuits.

FIG. 10 is a screenshot of a graphical user interface for generatingsimulated neural circuits.

FIG. 11 is a flow chart of a process for generating a simulated neuralcircuit.

FIG. 12 is a block diagram of a system for generating simulated neuralcircuits.

FIG. 13 is a flow chart of a process for wiring a simulated neuralcircuit.

DETAILED DESCRIPTION

Systems for generating or growing simulated neural circuits aredescribed below. In such systems, simulated neural circuits may begenerated by simulating the growth of the axonal and dendritic trees ofarrays of simulated neurons and by simulating the formation anddisintegration of connections (i.e., synapses) between neurons. In suchsystems, the growth of the dendritic trees of arrays of neurons, thegrowth of the axonal trees of arrays of neurons, and the wiring ofneurons to each other may be achieved via strictly local interactions.These interactions may be both between elements, for example between anaxonal segment of one neuron and a dendritic segment of the next, aswell as between an element and local properties of the environment withwhich it is in contact, for example between the tip of a growing axonand a “chemical” gradient of a substance which attracts or repulses thataxon as it grows. The interaction with chemical gradients controls thelong-distance trajectory of axons and dendrites, and hence establishesthe coarse fabric of the neural circuit. The interaction ongoing betweenactive elements wires the circuit with specificity that embedsinformation and function. Furthermore, the formation, growth, anddevelopment of simulated neural circuits may be activity-dependent,influenced by activity in the elements of the simulated neural circuitsas they grow.

Wiring a neural circuit can be seen as a choice of which neurons shouldbe connected out of all the possible combinations and how heavily suchconnections should be weighted. Therefore, it is a common property ofneural network models to predefine a pattern of connectivity, and thento strengthen or weaken the weights of extant connections to reduce thisinitial pattern to a final functional circuit. A weight of zero wouldcorrespond to an absent connection, while a relatively high weight wouldcorrespond to a strong connection from which an output generated inresponse to an input received at the connection would be weightedaccording to the weight assigned to the connection. However, to producea pattern of connectivity through setting weights to zero requires thatall potential connections between neurons be implemented in the initialpredefined pattern. This is difficult, if not impossible, forbiological-scale neural circuits because of the combinatorial explosionof potential connections.

For example, a set of 10,000 neurons, each with 1,000 synapticconnections spread across 100 dendritic branches, will have a total of10⁷ connections. However, if the axons of a second set of 10,000 neuronsmakes all-to-all possible connections with the dendrites of the firstset, there would be 10¹² all-to-all potential connections between thetwo sets of neurons. In other words, to produce a wiring pattern forthis example circuit using approaches based solely on strengthening orweakening existing weights would require 10⁵ greater usage ofcomputational resources than are needed for the final circuit.

However, as disclosed below, it is possible to efficiently wiresimulated neural circuits using techniques that do not rely onmaintaining and modifying the huge space of all potential connections.The strengthening and/or weakening of the weights of existingconnections may be compatible with the techniques for connection,formation, and dissolution disclosed below, but such strengtheningand/or weakening is not a necessary component of the techniques forgrowing and wiring simulated neural circuits disclosed below. In someimplementations, all connection weights may be uniform.

A set of local rules may be introduced to guide the appearance anddisappearance of connections during the wiring process. For example,simulated dendritic branches may form and subsequently stabilize andmaintain connections with simulated axons that fire in correlation withthe dendritic branches. Connections of a dendritic branch made withsimulated axons that have poorly correlated firing may be destabilizedand lost. Stability may be a state variable of the connection betweentwo neurons that can be increased or decreased as a function of therelationships of activity (e.g., temporal correlation) of the twoneurons. If stability drops below some parameterized value, theconnection may be removed. Additionally or alternatively, in order toensure that all dendrites receive a desired density of connections,simulated axons may be influenced to grow in the direction of and/orconnect to simulated dendrites that are impinged by relatively fewsimulated axons.

Three-dimensional voxel spaces are particularly well-suited forsimulating the physical space in which biological neurons grow. Theyfacilitate the efficient simulation of the local interaction betweengrowing neural elements as well as the processing of local rules thatguide the forming and disintegration of connections during the wiringprocess. Accordingly, in some implementations, simulated neural circuitsmay be grown within three-dimensional voxel spaces.

FIG. 1 a is a block diagram of a simulated neuron 100 at a first stageof growth. The simulated neuron 100 may be generated and thereaftergrown (as discussed in more detail below) by, for example, a system forgrowing simulated neural circuits such as that illustrated in FIG. 12and discussed more fully below. The simulated neuron 100 of FIG. 1 aincludes a cell body 102, or soma, and a dendritic tree 104 havingsegments 104(a), 104(b), and 104(c). Input signals to the simulatedneuron are received by the dendritic tree 104 and transmitted to thecell body 102. As illustrated in FIG. 1 a, the dendritic tree 104 hasonly a single branch. However, as the simulated neuron 100 grows, thedendritic tree 104 may recursively divide, resulting in numerousdendritic branches and sub-branches within the dendritic tree 104.

Attractants and Repellents with Cell Type-Specific Interactions to GuideCoarse-Grained Circuit Growth

The physical spaces in which biological neurons grow may includechemotropic substances (e.g., chemical attractants and/or repellants)that influence the direction or orientation of the growth of theneurons' dendrites. That is to say, as the dendritic branches of abiological neuron grow, these dendritic branches may be attracted towardone or more chemical attractants and/or repelled away from one or morechemical repellants present in the physical space. Such chemicalattractants and/or repellants may exhibit static spatial patterns (e.g.,gradients or stratifications) in the physical space and multiplechemical attractants and/or multiple chemical repellants may be presentin the same physical space concurrently. Further, a givenattractant/repellent may have a distinctly parameterized interactionwith the axonal and dendritic branches of each distinct type of neuron.

Additionally or alternatively, the spatial patterns of the chemicalattractants and/or repellants in the physical space may change overtime. For example, the firing of an axon of a neuron may be accompaniedby the release of a chemical attractant that may influence one or moredendritic branches to grow in the direction of the released chemicalattractant. In this manner, dendritic branches may be influenced to growtoward neural activity.

As the dendritic trees of neurons grow, the new branches that sproutfrom the dendritic trees may be of various different types that growdifferently and that interact differently with other neural elements.For example, different types of dendritic branches may exhibit differentaffinities for different chemotropic substances present in the physicalspace in which neurons grow such that different chemotropic substancesmay influence the growth of different types of dendritic branchesdifferently. Furthermore, different types of dendritic branches mayexhibit different affinities for different types of neural elements suchthat certain types of dendritic branches may have a greater tendency togrow towards and/or form connections with certain types of neuralelements than others.

The simulated neuron 100 of FIG. 1 a is grown in a simulated physicalspace 106 that, while not illustrated as such, may include one or morespatial patterns of simulated molecular attractants and/or one or morespatial patterns of simulated molecular repellants that may influencethe direction or orientation of growth of the simulated neuron 100.Consequently, as the dendritic tree 104 of the simulated neuron 100grows, its dendrites may be attracted toward the one or more simulatedmolecular attractants and/or repelled away from the one or moresimulated molecular repellants.

Branch Tips as Automata

In one implementation, axonal and dendritic branches may be grown by aseries of incremental actions taken by their “tip” elements. Thesebranch tips are automata that may exist in one of a plurality of states,with each state being associated with a particular action. For example,in one implementation, 5 states exist: terminated, extending,bifurcating, retracting, and static with permitted actions as follows:

Tip state Permitted action Terminated None Extending Extend a single tipextension, which becomes the active tip Bifurcating Extend two tipextensions, both of which become active tips Retracting Eliminate thecurrent tip, the parent becomes the active tip Static Do nothing butremain active

FIG. 1 b is a block diagram of the simulated neuron 100 of FIG. 1 aafter growing the active dendritic segment 104(c) according to thebifurcate growth state. As illustrated in FIG. 1 b, the active dendriticsegment 104(c) of FIG. 1 a has been bifurcated into two new activedendritic segments 104(c)(1) and 104(c)(2) and dendritic segment 104(c)has been deactivated.

Methods of Extension of a Branch Tip

In some implementations, growing an active dendritic segment accordingto the bifurcate growth state involves randomly, or pseudo randomly,generating a collection of candidate extension segments and thenselecting two or more segments from the collection of candidate segmentsas new segments to be added to the dendritic tree. For example, a set ofcandidate extension segments having different orientations (e.g.,different azimuthal angles) and/or lengths may be generated randomly bythe system. Constraints then may be imposed to limit the universe ofpotential candidate extension segments from which the actual candidateextension segments may be selected (e.g., a maximum azimuthal angle maybe defined for the potential candidate extension segments) and theactual segments to be added then may be selected.

After the candidate extension segments have been selected, eachcandidate segment may be assigned a score representing a measure of therelative desirability of adding the segment to the dendritic tree. Forinstance, a candidate segment that is oriented in the direction of asimulated attractant may be rewarded whereas a candidate segment that isoriented in the direction of a simulated repellant may be penalized.Similarly, and as discussed more fully below, a candidate dendriticsegment that enters an area of the physical space (e.g., a voxel) thatis crowded with one or more other dendritic branches may be penalized,whereas a candidate dendritic segment that enters an area of thephysical space that is not crowded by one or more other dendriticbranches may be rewarded. Further, a candidate dendritic segment thatenters an area of the physical space occupied by high density of axonsmay be rewarded, whereas a candidate dendritic segment that enters anarea of the physical space not occupied by axons may be penalized.

To implement a bifurcating extension, for example, after each candidatesegment has been assigned a score, two of the segments are selected tobe added. In some implementations, the selected segments may simply bethose with the highest scores. In alternative implementations, thecandidate segments may be first ranked according to their assignedscores, and then the two segments to be added to the dendritic tree 104may be selected randomly, or stochastically, with probabilities of beingchosen being proportional to the scores of each candidate extension.

FIG. 1 c is a block diagram of the simulated neuron 100 of FIG. 1 aafter growing the active dendritic segment 104(c) according to theextend growth state. As illustrated in FIG. 1 c, the active dendriticsegment 104(c) of FIG. 1 a has been deactivated and a new activedendritic segment 104(d) has been added to the end of the dendritic tree104.

Just as growing an active dendritic segment according to the “bifurcate”growth state, in some implementations, growing an active dendriticsegment according to the “extend” growth state involves randomly, orpseudo randomly, generating a collection of candidate extension segmentsand then selecting one candidate segment from this collection ofcandidate segments as the new segment to be added using the techniquesdescribed above.

FIG. 1 d is a block diagram of the simulated neuron 100 of FIG. 1 aafter growing the active dendritic segment 104(c) of FIG. 1 a accordingto the retract growth state. As illustrated in FIG. 1 d, the activedendritic segment 104(c) of FIG. 1 a has been deleted from the dendritictree 104 and the dendritic segment 104(b) immediately preceding theactive dendritic segment 104(c) of FIG. 1 a has been activated. Growingan active segment according to the “retract” growth state may be animportant capability in implementations in which multiple simulatedneurons are grown simultaneously, because it may enable a branch of asimulated neuron to withdraw from a region containing a poor selectionof branches with which to potentially form connections, and subsequentlyextend into another region containing a richer selection of brancheswith which to potentially form connections.

FIG. 1 e is a block diagram of the simulated neuron 100 of FIG. 1 aafter growing the active dendritic segment 104(c) of FIG. 1 a accordingto the terminate growth state. As illustrated in FIG. 1 e, the activedendritic segment 104(c) of FIG. 1 a has been deactivated, therebyeffectively terminating the growth of the dendritic tree 104.

FIG. 1 f is a block diagram of the simulated neuron of FIG. 1 a aftergrowing the active dendritic segment 104(c) of FIG. 1 a according to therest (or static) growth state. As illustrated in FIG. 1 f, after growingthe active dendritic segment 104(c) according to the rest growth state,the simulated neuron remains unchanged. That is to say, the activedendritic segment 104(c) of FIG. 1 a remains active and no additionalbifurcation, extension, or retraction has occurred.

FIG. 1 g is a block diagram of the simulated neuron 100 of FIG. 1 aafter the simulated neuron 100 has progressed through a series of growthstages. As illustrated in FIG. 1 g, the original dendritic tree 104illustrated in FIG. 1 a has extended and bifurcated many times overresulting in a more elaborate tree structure. In addition, the directionof the growth of the dendritic tree 104 has been in a generally upwarddirection. The generally upward direction of the growth of the dendritictree 104 is here due to the presence of a gradient (not illustrated) ofa simulated attractant in the upward direction of the simulated physicalspace 106.

The above discussion and FIGS. 1 a-1 g describe the growth of thedendritic tree 104 of the simulated neuron 100 as emanating from one ormore active segments located on the ends of the branches of thedendritic tree 104. However, in some implementations, active segmentsmay be located anywhere along the dendritic branches. Consequently, inthese implementations, new dendritic branches may sprout from any activesegment along a dendritic branch, not just from the end of a dendriticbranch. Furthermore, while the above discussion and FIGS. 1 a-1 gdescribe techniques for growing a dendritic tree of a simulated neuron,similar techniques may be applied to grow an axonal tree of a simulatedneuron.

Voxel Space: a Substrate for the Growth of Neural Circuitry

In some implementations, the simulated physical space in which simulatedneural circuits are grown may be modeled using a three-dimensional arrayof elements, or voxels, known as a voxel space. In such implementations,the individual voxels of the voxel space represent individual portionsof the simulated physical space. Voxels store a number of differentlocal variables that collectively describe the properties of theportions of the simulated physical space that the voxels represent. Forexample, voxels may be configured to store information about thedendrite segment density (i.e., the number and type of dendriticsegments present) and/or the axonal segment density (i.e., the numberand type of axonal segments present) in the portions of the simulatedphysical space represented by the voxels. In addition, voxels may beconfigured to store information about the presence and/or concentrationsof simulated attractants and/or simulated repellants in the portions ofthe simulated physical space represented by the voxels.

FIG. 1 h is a block diagram of the simulated neuron 100 of FIG. 1 g thatvisually illustrates implementing the simulated neuron 100 in a voxelspace 108. As illustrated in FIG. 1 h, the individual voxels of thevoxel space 108 are depicted as three-dimensional cubes. So as not toobscure other elements, only those individual voxels that are occupiedby dendritic branch segments are illustrated in FIG. 1 h. Nevertheless,it will be understood that the voxel space 108 is a three-dimensionalarray of voxels that includes a number of additional voxels that are notvisually depicted in FIG. 1 h.

A voxel space may be a particularly appropriate data structure forimplementing simulated neurons because voxel spaces are good analoguesto the physical spaces in which biological neurons grow. For example,due to their ability to store localized information, voxels may beparticularly appropriate for simulating the presence of repellantsand/or attractants locally released during growth by different simulatedneural elements. Furthermore, the ability of voxels to store localizedinformation also may limit the processing and/or computational loadinvolved in processing local rules introduced to guide the formation anddisintegration of connections between various different simulated neuralelements.

Moreover, using a voxel space to implement simulated neurons may limitthe processing and/or computational load involved in extending and/orbifurcating dendritic branches and axons. As described above, dendriticbranches and axons may be extended and/or bifurcated by selecting acollection of candidate extension segments, calculating and assigningdesirability scores to the selected candidate extension segments, andselecting one or more candidate extension segments based on the assigneddesirability scores.

Growing simulated neural circuits in a voxel space may enable an elegantand efficient process for calculating and assigning desirability scoresfor candidate extension segments. As discussed above, depending on thetype of neural element from which they extend, candidate extensionsegments may have different affinities for different types of neuralelements and/or different types of simulated attractants and repellants.For example, candidate segments may have negative affinities for certainchemical repellants and positive affinities for certain chemicalattractants. Similarly, candidate segments for a dendritic branch mayhave a negative affinity for other dendrites and a positive affinity foraxons, while candidate segments for axons may have a negative affinityfor other axons and a positive affinity for dendritic branches.

Because voxels enable the storage of localized information such as, forexample, chemical attractant concentrations, chemical repellantconcentrations, and neural element densities, a desirability score for acandidate extension segment may be calculated simply by considering thelocalized information stored in the voxel(s) into which the candidateextensions extend. For example, the desirability score, V, of acandidate extension segment may be defined algebraically by:

$\begin{matrix}{V = {{\sum\limits_{i}{a_{i} \cdot \left\lbrack c_{i} \right\rbrack}} + {\sum\limits_{j}{a_{j}^{10}\left\lbrack d_{j} \right\rbrack}}}} & \lbrack 1\rbrack\end{matrix}$

where a_(i) represents the affinity of the candidate extension segmentfor a particular chemical present in the voxel(s) into which thecandidate segment extends, c_(i) represents the concentration of aparticular chemical present in the voxel(s) into which the candidatesegment extends, a_(j) represents the affinity of the candidateextension segment for a particular type of neural element present in thevoxel(s) into which the candidate segment extends, and d_(j) representsthe density of a particular type of neural element present in thevoxel(s) into which the candidate segment extends. Equation 1 is merelyone example of an equation for calculating desirability scores forcandidate segment extensions. Accordingly, desirability scores can becalculated according to different equations that include differentand/or additional factors that influence growth.

FIG. 2 a is a screenshot of a graphical user interface (GUI) 200 forgrowing simulated neural circuits using, for example, a system forgrowing simulated neural circuits such as the system of growingsimulated neural circuits illustrated in FIG. 12 and discussed morefully below. The GUI 200 may be rendered on, for example, a display ormonitor. Among other features, the GUI 200 includes a setup control tab202, a simulation pane 204, and a simulate control tab 206.

More particularly, the setup control tab 202 includes, among otherfeatures, a voxel array setup control 202(a), a gradient setup control202(b), and a simulated neural circuit setup control 202(c). The voxelarray setup control 202(a) enables a user to specify the dimensions of avoxel space in which to grow a simulated neuron or a simulated neuralcircuit as well as the size (i.e., scale) of each voxel in the voxelspace. As illustrated in FIG. 2 a, the voxel array setup control 202(a)specifies a voxel space of one hundred voxels by fifty voxels by onehundred voxels, where each individual voxel is a five by five unit cube.

The gradient setup control 202(b) enables a user to specify one or moregradients of chemotropic substances (e.g., attractants or repellants) tobe generated within the voxel space. As illustrated in FIG. 2 a, thegradient setup control 202(b) enables a user to specify the magnitudesand orientations of up to three different three-dimensional gradients ofsimulated attractants and/or repellants (e.g., gradient A, gradient B,and/or gradient C). While the gradient setup control 202(b) illustratedin FIG. 2 a only enables a user to specify three gradients of simulatedattractants and/or repellants, additional gradients of attractantsand/or repellants may be generated. Furthermore, the spatial patterns ofsimulated attractants and/or repellants generated in the voxel space maynot be limited to gradients. Rather, other spatial patterns ofattractants and/or repellants, such as, for example, stratifications,also may be generated in the voxel space.

The simulated neural circuit setup control 202(c) enables a user tospecify the number and type of simulated neurons to be generated andgrown as well as the initial position of the cell bodies of thesimulated neurons to be generated and grown. As illustrated in FIG. 2 a,the simulated neural circuit setup control 202(c) specifies that onepyramidal cell-type simulated neuron is to be generated and grown.

The simulation pane 204 presents a visual representation of thesimulated neuron or simulated neural circuit generated and grown usingthe GUI 200. As illustrated in FIG. 2 a, the simulation pane 204includes an outline of a voxel space 204(a) and a visual representationof a simulated neuron 204(b). The dimensions of the voxel space 204(a)correspond to the dimensions specified for the voxel space in the voxelarray setup control 202(a). Similarly, one pyramidal cell-type simulatedneuron 204(b) is shown in the voxel space 204(a) because the simulatedneural circuit setup control 202(c) specifies that one pyramidalcell-type simulated neuron is to be generated in the voxel space 204(a).

The GUI 200 also includes a simulate tab 206 that enables a user to growthe simulated neuron or neural circuit generated using the setup tab202. FIG. 2 b is a screenshot so of the GUI 200 with the simulate tab206 selected. As illustrated in FIG. 2 b, the simulate tab 206 includes,among other features, a grow button 206(a) and a manual growth control206(b). Selection of the grow button 206(a) causes the system to growthe simulated neuron or neural circuit generated using the setup tab202. For example, selecting the grow button 206(a) may cause the systemto select a growth state (e.g., bifurcate, extend, retract, terminate,or rest) for each active segment of the simulated neuron or neuralcircuit and grow the simulated neuron or neural circuit according to theselected growth states. In some implementations, selection of the growbutton 206(a) may trigger the system to repeatedly progress throughgrowth stages until the grow button 206(a) is selected again, signalingthat growth of the simulated neuron or neural circuit should be halted.In other implementations, selection of the grow button 206(a) maytrigger only a single growth stage. In such implementations, repeatedgrowth of the simulated neurons 206(a) may be inspired through therepeated selection of the grow button 206(a).

The manual growth control 206(b) enables a user to manually control thegrowth of the simulated neuron or neural circuit. As illustrated in FIG.2 b, the manual growth control 206(b) enables a user to extend all ofthe active segments of the simulated neuron or neural circuit by aspecified number of segments, to retract all of the active segments ofthe simulated neuron or neural circuit by a specified number ofsegments, to bifurcate all of the active segments of the simulatedneuron or neural circuit, to terminate all of the active segments of thesimulated neuron or neural circuit, or to activate all of the previouslyterminated segments of the simulated neuron or neural circuit.

FIG. 2 c is a screenshot of the GUI 200 after the simulate tab 206 hasbeen used to grow the simulated neuron 204(b). As illustrated in FIG. 2c, the dendritic tree of the simulated neuron 204(b) has branched andextended many times as a result of the growth process.

FIG. 3 is a flowchart of a process 300 for growing a simulated neuron.The process 300 begins by generating a simulated neuron that has atleast one simulated dendritic branch (302). To grow the simulatedneuron, a growth state is selected for an active segment of thedendritic branch (304). For example, a particular growth state for theactive segment may be selected from among an extend growth state, abifurcate growth state, a retract growth state, a rest growth state, anda terminate growth state. In various implementations, the growth statemay be selected manually, randomly, pseudo-randomly, and/oralgorithmically.

After selecting the growth state for the active segment, the simulatedneuron is grown based on the selected growth state of the active segment(306). For example, growing the simulated neuron based on the “extend”growth state may involve extending the simulated dendritic branch.Growing the simulated neuron based on the “bifurcate” growth state mayinvolve bifurcating the simulated dendritic branch into at least two newactive segments, while growing the simulated neuron based on the“retract” growth state may involve retracting the simulated dendriticbranch, growing the simulated neuron according to the “rest” growthstate may involve maintaining the simulated dendritic branch in anunchanged state, and growing the simulated neuron according to theterminate growth state may involve deactivating the active segment.

After growing the simulated neuron according to the selected growthstate of the active segment, growth states may be selected for theactive segments (304) that resulted from growing the simulated neuronaccording to the previously selected growth state and the simulatedneuron may be grown again based on the newly selected growth states(306). In some implementations, the process of selecting growth statesfor active segments (304) and growing the simulated neuron according tothe selected growth states (306) may be repeated until the terminategrowth state has been selected for each active segment and each activesegment has been deactivated.

While FIG. 3 illustrates a process 300 for growing dendrites of asimulated neuron, a similar process may be used to grow the axons of asimulated neuron.

Biological neurons rarely grow in isolation. Rather, biological neuronsgenerally grow in groups. Consequently, as the biological neurons grow,they may interact with each other and compete with each other for space.The techniques for growing individual simulated neurons described abovemay be particularly powerful for growing multiple simulated neuronssimultaneously because they easily may be extended to account forconsiderations like crowding that are relevant to growing multiplesimulated neurons simultaneously.

FIG. 4 a is a block diagram of an array 400 of simulated neurons thatmay be generated and grown by, for example, a system for growingsimulated neural circuits such as the system for growing simulatedneural circuits illustrated in FIG. 12 and discussed more fully below.As illustrated in FIG. 4 a, the array 400 of simulated neurons is a fiveby five array 400 of simulated neurons and the simulated neurons are ata first stage of growth.

The simulated neurons of the array 400 of simulated neurons all may beof the same cell type or they may be of different cell types that growdifferently and that interact differently with different types of otherneural elements. For example, one type of simulated neuron may havedendritic branches that are strongly attracted to a particular simulatedattractant, while a different type of simulated neuron may havedendritic branches that are only weakly attracted (or not attracted atall) to the same simulated attractant. Additionally or alternatively,one type of simulated neuron may have dendritic branches that areattracted to a particular simulated attractant, whereas a different typeof simulated neuron may have dendritic branches that are repelled by thesame simulated attractant. Therefore, the cell types of the differentsimulated neurons and the distribution of the different simulatedattractants and/or repellants may influence the growth of the simulatedneurons.

Each of the simulated neurons of the array 400 of simulated neurons maybe grown according to the techniques discussed above, for example, inconnection with FIGS. 1 a-1 h, 2 a-2 c, and 3. Furthermore, the growthof each of the simulated neurons may be influenced by factors that arethe same as or similar to the factors discussed above, for example, inconnection with FIGS. 1 a-1 h, 2 a-2 c, and 3. One factor that may beparticularly important as multiple simulated neurons grow simultaneouslyand compete with each other for space may be dendritic branch density.

FIG. 4 b is a block diagram of the array 400 of simulated neurons thatillustrates the array 400 of simulated neurons at a second stage ofgrowth. As illustrated in FIG. 4 b, the dendritic trees of the simulatedneurons have branched and extended many times as a result of the growthprocess.

FIG. 5 a is a screenshot of the GUI 200 of FIGS. 2 a-2 c thatillustrates the use of the GUI 200 to generate and grow an array 204(c)of simulated neurons. As illustrated in FIG. 5 a, an array 204(c) ofsimulated neurons at a first growth state has been generated and ispresented in the simulation pane 204. As discussed above in connectionwith FIGS. 2 a-2 c, the simulate tab 206 of the GUI 200 is configured toenable a user to grow the simulated neurons of the array 204(c) ofsimulated neurons. For example, a user can grow the simulated neurons ofthe array 204(c) of simulated neurons using the grow button 206(a) orthe manual growth control 206(b).

FIG. 5 b is a screenshot of the GUI 200 after the simulate tab 206 hasbeen used to grow the array 204(c) of simulated neurons. As illustratedin FIG. 5 b, the dendritic trees of the simulated neurons have branchedand extended many times as a result of the growth process, therebycreating a relatively complex mesh of intertwined dendritic branches.

FIG. 6 is a flowchart of a process 600 for growing a simulated neuronalarray. The process 600 begins by generating an array of simulatedneurons having simulated dendritic branches (602). To grow the simulatedneurons, a growth state is selected for each active segment of thevarious simulated dendritic branches (604) and each simulated dendriticbranch is grown based on the growth state selected for its activesegment (606).

After growing the simulated dendritic branches according to the selectedgrowth states of the active segments, a new growth state may be selectedfor each active segment of the various simulated dendritic branches(604) and the simulated dendritic branches may be grown again based onthe newly selected growth states (606). In some implementations, theprocess of selecting a growth state for each active segment (604) andgrowing the simulated dendritic branches according to the selectedgrowth states (606) may be repeated until the terminate growth state hasbeen selected for each active segment and each active segment has beendeactivated.

While FIG. 6 illustrates a process for growing the dendritic branches ofan array of simulated neurons, a similar process may be used to grow theaxons of an array of simulated neurons.

A biological neural circuit generally includes numerous interconnectedneurons. In particular, the neurons are connected by forming connectionsbetween output axons and input dendritic branches. These connectionsenable signals to be communicated through the neural circuit from neuronto neuron. Like biological neural circuits, simulated neural circuitsmay be grown by simultaneously growing and connecting multiple simulatedneurons. The techniques for growing simulated neurons described abovemay be particularly powerful for growing simulated neural circuitsbecause they easily may be extended to accommodate the simultaneousgrowing and connecting of multiple simulated neurons.

FIG. 7 a is a block diagram of a four by six output array 702 ofsimulated neurons and a five by five input array 704 of simulatedneurons that may be generated and grown by, for example, a system forgrowing simulated neural circuits such as the system for growingsimulated neural circuits illustrated in FIG. 12 and discussed morefully below. As illustrated in FIG. 7 a, the output axons of the outputarray 702 of simulated neurons are hot connected to the input dendriticbranches of the input array 704 of simulated neurons.

However, connections between the output axons of the output array 702 ofsimulated neurons and the input dendrites of the input array 704 ofsimulated neurons may be created by growing both the output axons of theoutput array 702 of simulated neurons and the input dendritic branchesof the input array 704 of simulated neurons according to the processesdescribed above, for example, in connection with FIGS. 1 a-1 h, 2 a-2 c,3, 4 a, 4 b, 5 a, 5 b, and 6. As the output axons of the output array702 of simulated neurons and input dendrites of the input array 704 ofsimulated neurons grow, connections between the output axons and theinput dendrites may be formed.

Various conditions may influence the forming of connections betweenaxons and dendrites. In some implementations, connections may be formedbetween axons and dendrites that grow to within a certain proximity ofone another. For example, in implementations in which voxel spaces areused to simulate the physical space in which neurons grow, connectionsmay be formed between axons and dendrites that grow into the samevoxels. Alternatively, connections may be formed between axons anddendrites that grow to within a certain number of voxels of one another.For instance, connections may be formed between axons and dendrites thatgrow to within three voxels of one another.

The forming of connections between axons and dendrites also may beinfluenced by the cell types of the simulated neurons from which theaxons extend and the cell types of the simulated neurons from which thedendrites extend. For example, the axons that extend from different celltypes may have different affinities for dendrites that extend fromdifferent cell types and vice versa. Similarly, the forming ofconnections between axons and dendrites also may be influenced by axonalbranch type and dendritic branch type. For example, different axonalbranch types may have different affinities for different dendriticbranch types and vice versa.

FIG. 7 b is a block diagram of a simulated neural circuit 706 formed bygrowing and connecting the output axons of the output array 702 ofsimulated neurons and the input dendrites of the input array 704 ofsimulated neurons of FIG. 7 a. As illustrated in FIG. 7 b, theconnections between the output axons of the output array 702 ofsimulated neurons and the input dendrites of the input array 704 ofsimulated neurons enable signals to propagate from the output array 702of simulated neurons through the neural circuit 706 to the input array704 of simulated neurons.

Even though the output array 702 of simulated neurons includes onlytwenty four simulated output neurons and the input array 704 ofsimulated neurons includes only twenty five simulated input neurons, thesimulated neural circuit 706 formed by growing and connecting the outputaxons of the output array 702 of simulated neurons and the inputdendrites of the input array 704 of simulated neurons is relativelycomplex. Naturally, increasingly complex simulated neural circuits maybe grown by increasing the number of simulated neurons in the outputarray 762 of simulated neurons and/or increasing the number of simulatedneurons in the input array 704 of simulated neurons. For example,simulated neural circuits may be generated and grown in which the outputarray of simulated neurons includes thousands, hundreds of thousands, oreven millions of simulated output neurons and/or in which the inputarray of simulated neurons includes thousands, hundreds of thousands, oreven millions of simulated input neurons. Additionally or alternatively,simulated neural circuits may be generated in which there are more thantwo arrays of simulated neurons. For example, simulated neural circuitsmay be generated in which there are hundreds, thousands, or evenmillions of interconnected arrays of simulated neurons and suchsimulated neural circuits may include feedback, as well as feed-forward,connections between arrays.

As a result of the complexity of the simulated neural circuit 706 formedby growing and connecting the output axons of the output array 702 ofsimulated neurons and the input dendrites of the input array 704 ofsimulated neurons, many of the connections between output axons andinput dendrites are obscured in FIG. 7 b. For example, it is difficultto discern to which simulated neurons of the input array 704 individualneuron 702(a) of the output array 702 is connected. Similarly, it isdifficult to discern to which simulated neurons of the output array 702individual neuron 704(a) of the input array 704 is connected.

FIG. 7 c is a block diagram of the simulated neural circuit 706 thatillustrates individual neuron 702(a) of the output array 702 and theindividual neurons of the input array 704 to which individual neuron702(a) is connected in isolation. As illustrated in FIG. 7 c, individualneuron 702(a) of the output array 702 is connected to fourteen of thesimulated neurons of the input array 704. As further illustrated in FIG.7 c, multiple connections may exist between the individual simulatedneuron 702(a) and an individual simulated neuron of the input array 704.For example, the individual neuron 702(a) is connected to individualneuron 704(b) at both connection 708 and connection 710.

FIG. 7 d is a block diagram of the simulated neural circuit 706 thatillustrates individual neuron 704(a) of the input array 704 and theindividual neurons of the output array 702 to which individual neuron704(a) is connected in isolation. As illustrated in FIG. 7 d, individualneuron 704(a) of the input array is connected to eight of the simulatedneurons of the output array 702.

FIG. 8 a is a screenshot of a GUI 800 for growing simulated neuralcircuits using, for example, a system for growing simulated neuralcircuits such as the system for growing simulated neural circuitsillustrated in FIG. 12 and discussed more fully below. Like the GUI 200of FIGS. 2 a-2 c, GUI 800 includes, among other features, a setupcontrol tab 802, a simulation pane 804, and a simulate tab 806.

More particularly, the setup control tab 802 includes, among otherfeatures, a voxel array setup control 802(a), a gradient setup control802(b), and a simulated neural circuit setup control 802(c). The voxelarray setup control 802(a) enables a user to specify the dimensions of avoxel space in which to grow a simulated neural circuit as well as thesize (i.e., scale) of each voxel in the voxel space. As illustrated inFIG. 8 a, the voxel array setup control 802(a) specifies a voxel spaceof one hundred voxels by fifty voxels by one hundred voxels for growinga simulated neural circuit, with each voxel being a five by five unitcube.

The gradient setup control 802(b) enables a user to specify one or moregradients of simulated chemotropic substances (e.g., attractants orrepellants) to be generated within the voxel space. As illustrated inFIG. 8 a, the gradient setup control 802(b) enables a user to specifythe magnitudes and orientations of up to three differentthree-dimensional gradients of simulated attractants and/or repellents(e.g., gradient A, gradient B, and/or gradient C). While the gradientsetup control 802(b) illustrated in FIG. 8 a only enables a user tospecify three gradients of simulated attractants and/or repellants, insome implementations, additional gradients of attractants and/orrepellants may be generated. Furthermore, in some implementations, thespatial patterns of simulated attractants and/or repellants generated inthe voxel space may not be limited to gradients. Rather, other spatialpatterns of attractants and/or repellants, such as, for example,stratifications, also may be generated in the voxel space.

The simulated neural circuit setup control 802(c) includes an outputarray setup control 802(c)(1) that enables a user to specify the numberof rows and columns as well as the initial positions of simulated outputneurons for an output array of simulated neurons. In addition, thesimulated neural circuit setup control 802(c) includes an input arraysetup control 802(c)(2) that enables a user to specify the number ofrows and columns as well as the initial positions of simulated inputneurons for an input array of simulated neurons. As illustrated in FIG.8 a, the output array setup control 802(c)(1) specifies that the outputarray of simulated neurons is to include four rows and six columns ofsimulated output neurons and the input array setup control 802(c)(2)specifies that the input array of simulated neurons is to include fiverows and five columns of simulated input neurons.

The simulation pane 804 presents a visual representation of thesimulated neural circuit 804(a) specified by the parameters entered inthe setup tab 802. More particularly, the simulation pane 804 includes avisual representation of an output array 804(b) of simulated outputneurons and an input array 804(c) of simulated input neurons. Asillustrated in FIG. 8 a, the output array 804(b) of simulated neurons isa four by six array 804(b) of simulated output neurons as specified inthe output array setup control 802(c)(1). Similarly, the input array804(c) of simulated neurons is a five by five array 804(c) of simulatedinput neurons as specified in the input array setup control 802(c)(2).

As with GUI 200 of FIGS. 2 a-2 c, the GUI 800 also includes a simulatetab 806 that enables a user to grow and connect the axons of the outputarray 804(b) and the dendrites of the input array 804(c) to form asimulated neural circuit. FIG. 8 b is a screenshot of the GUT 800 withthe simulate tab 806 selected. As illustrated in FIG. 8 b, the simulatetab 806 includes, among other features, a grow button 806(a) and amanual growth control 806(b). Selection of the grow button 806(a) causesthe axons of the output array 804(b) and the dendrites of the inputarray 804(c) to grow. For example, when the grow button 806(a) isselected, a growth state (e.g., bifurcate, extend, retract, terminate,or rest) may be selected for each active axonal segment of the outputarray 804(b) and each active dendritic segment of the input array804(c). Thereafter, each active axonal segment and each active dendriticsegment may be grown according to the selected growth states. In someimplementations, selection of the grow button 806(a) may trigger arepeated progression through growth stages until the grow button 806(a)is selected again to signal that growth of simulated neural circuit804(a) should be halted. In other implementations, selection of the growbutton 806(a) may trigger only a single growth stage. In suchimplementations, repeated growth stages may be inspired through therepeated selection of the grow button 806(a).

The manual growth control 806(b) enables a user to manually control thegrowth of the simulated neural circuit 804(a). As illustrated in FIG. 8b, the manual growth control 806(b) enables a user to extend all of theactive axonal and dendritic segments by a specified number of segments,to retract all of the active axonal and dendritic segments by aspecified number of segments, to bifurcate all of the active axonal anddendritic segments, to terminate all of the active axonal and dendriticsegments, or to activate all of the previously terminated axonal anddendritic segments.

FIG. 8 c is a screenshot of the GUI 800 after the simulate tab 806 hasbeen used to grow the simulated neural circuit 804(a). As illustrated inFIG. 8 c, the axons of the output array 804(b) of simulated neurons andthe dendrites of the input array 804(c) of simulated neurons have beengrown and connected to complete the simulated neural circuit 804(a) suchthat signals may be communicated from the output array 804(b) ofsimulated neurons to the input array 804(c) of simulated neurons throughthe connections between the output axons and the input dendrites.

FIG. 9 a is a screenshot of the GUI 800 that illustrates the propagationof signals through the simulated neural circuit 804(a). For the purposesof illustration, active neural elements are depicted using light shadingand inactive neural elements are depicted using dark shading. Asillustrated in FIG. 9 a, simulated output neurons 804(b)(1), 804(b)(2),804(b)(3), and 804(b)(4) are active. Output signals generated by theactive output neurons 804(b)(1), 804(b)(2), 804(b)(3), and 804(b)(4) aretransmitted along their output axons and communicated to the simulatedneurons of the input array 804(c) through connections between the outputaxons of the active output neurons 804(b)(1), 804(b)(2), 804(b)(3), and804(b)(4) and dendrites of the simulated neurons of the input array804(c).

If the concurrent activity in the axons connected to a particulardendrite is sufficiently high, the dendrite may fire. For example, insome implementations, a dendrite may fire in response to the sum of themagnitudes of the activity levels in the axons connected to the dendriteexceeding a threshold level or in response to a threshold number of theaxons connected to the dendrite being active concurrently or within acertain time period. Additionally or alternatively, a dendrite may firein response to a non-linear pattern or combination of concurrentactivity in the axons connected to the dendrite.

The firing of a dendrite results in a signal being transmitted along thedendrite to the cell body of the simulated neuron to which the dendritebelongs. If the activity in the dendrites of a simulated neuron issufficiently high, the neuron may fire. For example, in someimplementations, a simulated neuron may fire in response to the sum ofthe magnitudes of the activity levels in the simulated neuron'sdendrites exceeding a threshold level or in response to a thresholdnumber of the simulated neuron's dendrites being active concurrently orwithin a certain time period. Additionally or alternatively, a simulatedneuron may fire in response to a non-linear pattern or combination ofconcurrent activity in the simulated neuron's dendrites.

FIGS. 9 b and 9 c illustrate the response of an individual simulatedneuron to activity in the axons connected to the simulated neuron'sdendrites in greater detail. In particular, FIGS. 9 b and 9 c are blockdiagrams of an isolated simulated input neuron 950 of a simulated neuralcircuit (not illustrated) that is connected to a number of simulatedoutput neurons 952 by connections between the input dendrites of thesimulated input neuron 950 and the output axons of the simulated outputneurons 952.

FIG. 9 b illustrates the simulated input neuron 950 firing in responseto activity in the axons connected to the dendrites of the neuron 950.As illustrated in FIG. 9 b, simulated output neurons 952(a), 952(b), and952(c) are active and therefore transmit output signals along theiroutput axons. The output signals in the output axons of simulated outputneurons 952(a), 952(b), and 952(c) generate activity in dendrites950(a), 950(b), 950(c), and 950(d), and the activity in dendrites950(a), 950(b), 950(c), and 950(d), in turn, causes simulated neuron 950to fire.

In contrast to FIG. 9 b, FIG. 9 c illustrates the simulated input neuron950 remaining inactive in response to activity in the axons connected tothe dendrites of the neuron 950. As illustrated in FIG. 9 c, the onlysimulated output neuron that is active is simulated output neuron952(d). Therefore, only the output axon of output neuron 952(d)transmits an output signal to the dendrites of the simulated inputneuron 950 and only dendrite 950(e) fires in response. Consequently, theactivity in the dendrites of the simulated input neuron 950 isinsufficient to trigger the simulated input neuron 950 to fire and thesimulated input neuron 950 remains inactive.

Referring again to FIG. 9 a, the output signals generated by the activesimulated output neurons 804(b)(1), 804(b)(2), 804(b)(3), and 804(b)(4)are communicated through the simulated neural circuit 804(a) along theoutput axons of the active simulated neurons 804(b)(1), 804(b)(2),804(b)(3), and 804(b)(4) and trigger the firing of numerous inputdendrites of the simulated input neurons of the input array 804(c). Inturn, the firing of the input dendrites results in the firing ofsimulated input neurons 804(c)(1), 804(c)(2), 804(c)(3), 804(c)(4),804(c)(5), and 804(c)(6). That is to say, the propagation of the signalsgenerated by the active simulated output neurons 804(b)(1), 804(b)(2),804(b)(3), and 804(b)(4) through the simulated neural circuit 804(a)results in the firing of simulated input neurons 804(c)(1), 804(c)(2),804(c)(3), 804(c)(4), 804(c)(5), and 804(c)(6).

In some implementations, activity in a simulated neural circuitgenerated in response to the activation of the simulated output neuronsof the output array of the simulated neural circuit influences thegrowth and development of the simulated neural circuit. For example, asdescribed further below, activity in the neural circuit may influencethe formation, stabilization, destabilization, and dissolution ofconnections between neural elements. Additionally or alternatively, insome implementations, the output axons of the simulated neurons of theoutput array are configured to release a simulated attractant inresponse to firing. Therefore, the concentration of simulatedattractants will be greater in the vicinity of frequently active axonsthan in the vicinity of infrequently active axons. Accordingly,dendrites of the simulated neurons of the input array may be stronglyattracted to frequently active axons. As a result, dendrites may growtoward and form connections with frequently active axons more often thandendrites grow toward and form connections with infrequently activeaxons. Consequently, in such implementations, the growth of simulatedneural circuits may be said to be activity dependent.

Additionally, the strength of the connections between axons anddendrites in a simulated neural circuit also may be influenced byactivity in the simulated neural circuit. As described above, thestrengthening and weakening of the weights of connections is compatiblewith the mechanisms described for maintaining and destabilizing synapticcontacts, but is not necessary for wiring the neural circuit. Forexample, when both an axon and a dendritic branch to which the axon isconnected fire in response to a particular signal, the connectionbetween the axon and the dendritic branch may be strengthened, wherestrengthening of the connection means that future activity in the axonhas a relatively larger impact on activity generated in the dendriticbranch in response. Similarly, when the firing of an axon and adendritic branch to which the axon is connected triggers the firing ofthe cell body to which the dendritic branch belongs, the connectionbetween the axon and the dendritic branch may be strengthened.

Connections between axons and dendritic branches also may be weakened asa result of activity in the simulated neural circuit. For example, whenan axon fires in response to a signal, but a dendritic branch to whichthe axon is connected does not fire in response to the same signal, theconnection between the axon and the dendritic branch may be weakened,where weakening of the connection means that future activity in the axonhas a relatively smaller impact on activity generated in the dendriticbranch in response. Similarly, if an axon does not fire but a dendriticbranch to which the axon is connected and/or the cell body to which thedendritic branch belongs does fire, the connection between the axon andthe dendritic branch may be weakened. Likewise, if the firing of an axondoes not trigger the firing of a dendritic branch to which the axon isconnected, but the cell body to which the dendritic branch belongs firesnevertheless, the connection between the axon and the dendritic branchmay be weakened.

In this manner, connections that result in the propagation of a signalthrough a simulated neural circuit (e.g., from axon, to dendriticbranch, to cell body) may be strengthened and connections that do notresult in the propagation of a signal through a simulated neural circuitmay be weakened.

FIG. 10 is a screenshot of a GUI 1000 for growing a simulated neuralcircuit using, for example, a system for growing simulated neuralcircuits such as the system for growing simulated neural circuitsillustrated in FIG. 12 and discussed more fully below. Like the GUIs 200and 800 of FIGS. 2 a-2 c and 8 a-8 c, GUI 1000 includes, among otherfeatures, a setup control tab 1002, a simulate tab 1003, a display tab1004, and a parameters tab 1005.

The setup control tab 1002 of GUI 1000 enables a user to specify theinitial components of a simulated neural circuit to be grown as well asthe properties of the environment in which the simulated neural circuitwill be grown and includes, among other features, a voxel array setupcontrol 1002(a), a gradient setup control 1002(b), and a simulatedneural circuit setup control 1002(c). The voxel array setup control1002(a) enables a user to specify the dimensions of a voxel space inwhich to grow a simulated neural circuit as well as the size (i.e.,scale) of each voxel in the voxel space. The gradient setup control1002(b) enables a user to specify one or more three-dimensionalgradients of simulated chemotropic substances to be generated within thevoxel space. The simulated neural circuit setup control 1002(c) includesan output array setup control 1002(c)(1) that enables a user to specifythe number of rows and columns as well as the initial positions ofsimulated output neurons for an output array of simulated neurons. Inaddition, the simulated neural circuit setup control 1002(c) includes aninput array setup control 1002(c)(2) that enables a user to specify thenumber of rows and columns as well as the initial positions of simulatedinput neurons for an input array of simulated neurons.

A user can toggle between display tab 1004 and parameters tab 1005. Whendisplay tab 1004 is selected, a simulation pane (not shown) similar tothe simulation panes 204 and 804 of FIGS. 2 a-2 c and 8 a-8 c isdisplayed. When parameters tab 1005 is selected, various controls aredisplayed that enable a user to specify different properties of thesimulated neural circuit to be grown using GUI 1000 and initializedusing setup control tab 1002. For example, when parameters tab 1005 isselected, a contact properties control 1006 is displayed that enables auser to specify the extent to which activity in the simulated neuralcircuit influences the stability of connections between axons anddendrites in the simulated neural circuit. In addition, a branchproperties control 1008 that enables a user to specify various differentproperties of dendritic branches of the simulated neural circuit isdisplayed when parameters tab 1005 is selected.

As illustrated in FIG. 10, the contact properties control 1006 includeseight fields, each corresponding to one of the eight possiblecombinations of axon, branch, and neuron activity: a no activity field1006(a), an active neuron field 1006(b), an active branch field 1006(c),an active branch and active neuron field 1006(d), an active axon field1006(e), an active axon and active neuron field 1006(f), an active axonand active branch field 1006(g), and an active axon, active branch, andactive neuron field 1006(h). These fields may be leveraged to define howactivity in the simulated neural circuit impacts the stability ofconnections within the simulated neural circuit. As described above,stability may be a state variable of the connection between two elementsthat can be increased or decreased as a function of the relationships ofactivity (e.g., temporal correlation) of the two elements. If stabilitydrops below some parameterized value, the connection between theelements may be removed.

The no activity field 1006(a) defines how the stability of a connectionbetween an axon and a dendritic branch should be impacted when neitherthe axon nor the dendritic branch nor the neuron to which the dendriticbranch belongs fire. As illustrated in FIG. 10, the stability of theconnection between an axon and a dendritic branch is to remain unchangedwhen the neither the axon nor the dendritic branch nor the neuron towhich the dendritic branch belongs fire.

The active neuron field 1006(b) defines how the stability of aconnection between an axon and a dendritic branch should be impactedwhen neither the axon nor the dendritic branch fire but the neuron towhich the dendritic branch belongs fires. As illustrated in FIG. 10, thestability of the connection between an axon and a dendritic branch is tobe decremented by two when neither the axon nor the dendritic branchfire but the neuron to which the dendritic branch belongs fires.

The active branch field 1006(c) defines how the stability of aconnection between an axon and a dendritic branch should be impactedwhen the dendritic branch fires but neither the axon nor the neuron towhich the dendritic branch belongs fires. As illustrated in FIG. 10, thestability of the connection between an axon and a dendritic branch is tobe decremented by one when the dendritic branch fires but neither theaxon nor the neuron to which the dendritic branch belongs fires.

The active branch and active neuron field 1006(d) defines how thestability of a connection between an axon and a dendritic branch shouldbe impacted when the dendritic branch and the neuron to which thedendritic branch belongs fire but the axon does not fire. As illustratedin FIG. 10, the stability of the connection between an axon and adendritic branch is to be decremented by two when the dendritic branchand the neuron to which the dendritic branch belongs fire but the axondoes not fire.

The active axon field 1006(e) defines how the stability of a connectionbetween an axon and a dendritic branch should be impacted when the axonfires but neither the dendritic branch nor the neuron to which thedendritic branch belongs fires. As illustrated in FIG. 10, the stabilityof the connection between an axon and a dendritic branch is to bedecremented by one when the axon fires but neither the dendritic branchnor the neuron to which the dendritic branch belongs fire.

The active axon and active neuron field 1006(f) defines how thestability of a connection between an axon and a dendritic branch shouldbe impacted when both the axon and the neuron to which the dendriticbranch belongs fire but the dendritic branch does not fire. Asillustrated in FIG. 10, the stability of the connection between an axonand a dendritic branch is to be decremented by one when both the axonand the neuron to which the dendritic branch belongs fire but thedendritic branch does not fire.

The active axon and active branch field 1006(g) defines how thestability of a connection between an axon and a dendritic branch shouldbe impacted when both the axon and the dendritic branch fire but theneuron to which the dendritic branch belongs does not fire. Asillustrated in FIG. 10, the stability of the connection between an axonand a dendritic branch is to be strengthened by four when both the axonand the dendritic branch fire but the neuron to which the dendriticbranch belongs does not fire.

The active axon, active branch, and active neuron field 1006(h) defineshow the stability of a connection between an axon and a dendritic branchshould be impacted when the axon, the dendritic branch, and the neuronto which the dendritic branch belongs all fire. As illustrated in FIG.10, the stability of the connection between an axon and a dendriticbranch is to be strengthened by eight when the axon, the dendriticbranch, and the neuron to which the dendritic branch belongs all fire.

Of course, the values specified in the no activity field 1006(a), theactive neuron field 1006(b), the active branch field 1006(c), the activebranch and active neuron field 1006(d), the active axon field 1006(e),the active axon and active neuron field 1006(f), the active axon andactive branch field 1006(g), and the active axon, active branch, andactive neuron field 1006(h) are not limited to those values illustratedin FIG. 10. Rather, any value may be specified in each of these fields.

Among other features, the branch properties control 1008 includes anaffinity definition control 1008(a) that enables a user to specifydendritic branch affinities for different simulated chemotropicsubstances and/or neural elements as the simulated neural circuit grows.For example, the affinity definition control 1008(a) includes gradient Aaffinity definition control 1008(a)(1), gradient B affinity definitioncontrol 1008(a)(2), and gradient C affinity definition control1008(a)(3), each of which enables a user to specify a polarity (i.e.,attractive or repulsive) and relative magnitude of dendritic branchaffinity for a corresponding gradient of a simulated chemotropicsubstance initialized using the gradient setup control 1002(b). Asillustrated in FIG. 10, gradient A definition control 1008(a)(1)specifies that the dendritic branches of the simulated neural circuitare attracted to the simulated chemotropic substance of gradient A witha relative magnitude of 1.

In addition, the affinity definition control 1008(a) also includes axonaffinity definition control 1008(a)(4) and dendrite affinity definitioncontrol 1008(a)(5) which enable a user to specify polarities andrelative magnitudes of dendritic branch affinities for axons anddendrites respectively. As illustrated in FIG. 10, axon affinitydefinition control 1008(a)(4) specifies that the dendritic branches ofthe simulated neural circuit are attracted to axons of the simulatedneural circuit with a relative magnitude of 1 and dendritic affinitydefinition control 1008(a)(5) specifies that dendritic branches of thesimulated neural circuit are repelled by other dendritic branches of thesimulated neural circuit with a relative magnitude of 1.

Additional and/or more specific dendritic branch affinity controls alsomay be presented to a user. For example, controls that enable a user tospecify affinities of specific types of dendritic branches for differentchemotropic substances and/or neural elements may be presented.Additionally or alternatively, controls that enable a user to specifyaffinities of dendritic branches that extend from specific types of cellbodies for different chemotropic substances and/or neural elements maybe presented.

In some implementations, controls that enable a user to specifyaffinities of axons or other neural elements for different chemotropicsubstances and/or neural elements also may be presented.

FIG. 11 is a flow chart of a process 1100 for growing a simulated neuralcircuit. The process 1100 begins by generating a first array ofsimulated neurons having input dendritic trees (1102(a)) and generatinga second array of simulated neurons having output axons (1102(b)).Thereafter, the input dendritic trees of one or more of the simulatedneurons of the first array are grown (1104(a)) and the output axons ofone or more of the second array of simulated neurons are grown(1104(b)). In some instances, as the input dendritic trees and theoutput axons are grown, individual dendritic branches of the dendritictrees and individual axonal branches of the output axons may growtowards each other. When individual dendritic branches and individualaxonal branches grow within a vicinity of each other, they may beselectively connected to each other to form a neural circuit (1106).

FIG. 12 is a block diagram of an exemplary system 1200 for growingsimulated neural networks. Briefly, the system includes a computer 1202,a display 1204, a keyboard 1206, and a pointing device 1208 (e.g., amouse).

The computer 1202 may be implemented by, for example, a general purposecomputer capable of responding to and executing instructions in adefined manner, a personal computer, a special-purpose computer, aworkstation, a server, a notebook or laptop computer, a personal digitalassistant (PDA), a wireless telephone, a device, a component, otherequipment, or some combination of these items that is capable ofresponding to and executing instructions. Particular implementations maynot include the display 1204, the keyboard 1206 and/or the pointingdevice 1208.

As illustrated in FIG. 12, computer 1200 includes one or more processors1202(a) that process instructions, one or more memory/storage devices1202(b) that store data and various programs such as operating systemsand application programs, and one or more input/output (I/O) interfaces1202(c) that enable computer 1202 to communicate with one or more otherdevices, such as, for example, display 1204, keyboard 1206, pointingdevice 1208, and/or a communications network (e.g., a local area network(LAN), a wide area network (WAN), or the Internet (not shown)).

The processor 1202(a) may be configured to receive instructions from,for example, a software application, a program, a piece of code, adevice, a computer, a computer system, or a combination thereof, whichindependently or collectively direct operations, as described herein.The instructions may be embodied permanently or temporarily in any typeof machine, component, equipment, storage medium, or propagated signalthat is capable of being delivered to the processor 1202(a).

Memory 1202(b) may include a volatile memory device such as a randomaccess memory (RAM) for storing computer instructions and data forprocessing by processor 1202(a). In addition, memory 1202(b) also mayinclude a non-volatile memory device such as a read-only memory (ROM)for storing invariant low-level system code or data for basic systemfunctions such as basic I/O, startup, or reception of keystrokes fromkeyboard 1206. Furthermore, memory 1202(b) may store computer executableinstructions for an operating system and/or application programs,including, for example, applications for growing simulated neuralcircuits, as well as data files. During operation, computer executableinstructions may be loaded into a region of RAM in memory 1202(b) sothat they may be accessed by processor 1202(a) in order to executesoftware programs.

I/O interfaces 1202(c) may include a display interface that enablescomputer 1202 to render graphics, images, and/or text on display 1204.In addition, I/O interfaces 1202(c) may include a keyboard interfacethat enables computer 1202 to receive keystroke input from keyboard1206, a pointing device interface that enables computer 1202 to receiveinput from pointing device 108, and a communications interface thatenables computer 1202(c) to exchange data or other information with acommunications network (not shown).

The system 1200 for growing simulated neural circuits of FIG. 12 ismerely an example of a system for growing simulated neural circuits.Accordingly, other systems, configurations, and/or architectures alsomay be used to implement systems for growing simulated neural circuits.

As described above, the introduction of a set of local rules that guidethe appearance and disappearance of connections in a simulated neuralcircuit based on activity in the simulated neural circuit may enable anelegant process for wiring the simulated neural circuit withoutrequiring that all of the potential connections between the elements ofthe neural circuit be implemented in an initial predefined pattern.

By way of summary, FIG. 13 is a flow chart of a process 1300 for wiringa simulated neural circuit by using rules that guide the appearance anddisappearance of connections in a simulated neural circuit based onactivity in the simulated neural circuit. As described above, thesimulated neural circuit may include a number of simulated neurons thathave input branches that are configured to enable the formation ofconnections with different inputs. In addition, the input branches ofthe simulated neurons may be configured to activate in response toactivity in the inputs to which the input branches are connected.Similarly, the simulated neurons may be configured to activate inresponse to activity in their input branches.

The process 1300 begins by selectively forming initial connectionsbetween various input branches and various inputs (1302). For example,the input branches and/or the various inputs may be grown, andconnections may be formed between input branches and inputs that grow towithin a certain proximity of each other.

Eventually, activity is received and/or generated in the inputs (1304)resulting in a pattern of activity that propagates through the inputbranches and that ultimately may result in the firing of one or more ofthe simulated neurons. As this pattern of activity propagates throughthe input branches and simulated neurons of the simulated neuralcircuit, the stability of the connections between input branches andinputs is moderated (e.g., increased or decreased) based on the patternof activity generated in the input branches and simulated neurons(1306).

In some implementations, a set of local rules may define how thestability of connections between input branches and inputs is impactedby the pattern of activity generated in the input branches and simulatedneurons. For example, the rules outlined in the following table maydefine whether to increase or reduce the stability of a connectionbetween a particular input branch and a particular input:

Reduce stability of the input was not activated, the input branchremained inactive in connection between response to the activated set ofinputs, and the simulated neuron to input branch and which the inputbranch belongs activated in response to the activated input when: set ofinputs the input was activated, the input branch was activated inresponse to the activated set of inputs, and the simulated neuron towhich the input branch belongs remained inactive in response to theactivated set of inputs the input was not activated, the input branchwas activated in response to the activated set of inputs, and thesimulated neuron to which the input branch belongs was activated inresponse to the activated set of inputs the input was activated, theparticular input branch remained inactive in response to the activatedset of inputs, and the simulated neuron to which the input branchbelongs remained inactive in response to the activated set of inputs theinput was among the activated set of inputs, the input branch remainedinactive in response to the activated set of inputs, and the simulatedneuron to which the input branch belongs remained inactive in responseto the activated set of inputs Increase stability of the input was amongthe activated set of inputs, the input branch was connection betweenactivated in response to the activated set of inputs, and the simulatedinput branch and neuron to which the input branch belongs remainedinactive in input when: response to the activated set of inputs theinput was among the activated set of inputs, the input branch wasactivated in response to the activated set of inputs, and the simulatedneuron to which the input branch belongs was activated in response tothe activated set of inputs

In order to complete the wiring of the simulated neural circuit, theinputs repeatedly may be subjected to different patterns of activity(1304) and the connections between input branches and inputs may bemoderated continually based on the resultant patterns of activity in theinput branches and simulated neurons (1306). As the process progresses,the wiring of the circuit may evolve, with new connections being formedand existing connections disintegrating when their stability drops belowsome parameterized value.

The systems and/or techniques for growing simulated circuits describedabove may be used to design simulated neural circuits for real-lifeapplications. For example, after growing a simulated neural circuitusing the systems and/or techniques described above, the grown simulatedneural circuit may be implemented in digital logic or fabricated in anintegrated circuit (IC) for use in processing or other applications.Furthermore, employing the systems and techniques for growing simulatedneural circuits described above to simulate the growth of biologicalneurons may enable researchers and scientists to develop a betterunderstanding of the mechanisms that guide and control the growth ofbiological neurons.

The systems and techniques described above are not limited to anyparticular hardware or software configuration. Rather, they may beimplemented using hardware, software, or a combination of both. Inaddition, the methods and processes described may be implemented ascomputer programs that are executed on programmable computers comprisingat least one processor and at least one data storage system. Thecomputer programs may be implemented in a high-level compiled orinterpreted programming language, or, additionally or alternatively, thecomputer programs may be implemented in assembly or other lower levellanguages, if desired. Such computer programs typically will be storedon computer-usable storage media or devices (e.g., CD-Rom, RAM, ormagnetic disk). When read into a processor of a computer and executed,the instructions of the programs may cause a programmable computer tocarry out the various operations described above.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications and implementations may bemade. For example, while techniques for growing simulated neuralcircuits have been described largely in the context of simultaneouslygrowing and interconnecting arrays of simulated neurons, the simulatedneurons that compose simulated neural circuits need not be configured inarrays. Furthermore, useful results still may be achieved if steps ofthe disclosed techniques are performed in a different order and/or ifcomponents in the disclosed systems are combined in a different mannerand/or replaced or supplemented by other components. Accordingly, otherimplementations are within the scope of the following claims.

1. A method for moderating connections in a simulated neural circuit,wherein the simulated neural circuit includes a plurality of simulatedneurons having input branches that are configured to connect to aplurality of inputs and activate in response to activity in the inputsto which the input branches are connected, and wherein the simulatedneurons are configured to activate in response to activity in theirinput branches, the method comprising: forming initial connectionsbetween various input branches and various inputs; activating a set ofthe inputs; and moderating the stability of connections between inputbranches and inputs based on the activated set of inputs and a patternof activity generated in the input branches and simulated neurons inresponse to the activated set of inputs.
 2. The method of claim 1wherein moderating the stability of connections between input branchesand inputs includes reducing the stability of a connection between aparticular input branch and a particular input when the particular inputwas not among the activated set of inputs, the particular input branchremained inactive in response to the activated set of inputs, and thesimulated neuron to which the particular input branch belongs wasactivated in response to the activated set of inputs.
 3. The method ofclaim 1 wherein moderating the stability of connections between inputbranches and inputs includes reducing the stability of a connectionbetween a particular input branch and a particular input when theparticular input was not among the activated set of inputs, theparticular input branch was activated in response to the activated setof inputs, and the simulated neuron to which the particular input branchbelongs remained inactive in response to the activated set of inputs. 4.The method of claim 1 wherein moderating the stability of connectionsbetween input branches and inputs includes reducing the stability of aconnection between a particular input branch and a particular input whenthe particular input was not among the activated set of inputs, theparticular input branch was activated in response to the activated setof inputs, and the simulated neuron to which the particular input branchbelongs was activated in response to the activated set of inputs.
 5. Themethod of claim 1 wherein moderating the stability of connectionsbetween input branches and inputs includes reducing the stability of aconnection between a particular input branch and a particular input whenthe particular input was among the activated set of inputs, theparticular input branch remained inactive in response to the activatedset of inputs, and the simulated neuron to which the particular inputbranch belongs remained inactive in response to the activated set ofinputs.
 6. The method of claim 1 wherein moderating the stability ofconnections between input branches and inputs includes reducing thestability of a connection between a particular input branch and aparticular input when the particular input was among the activated setof inputs, the particular input branch was activated in response to theactivated set of inputs, and the simulated neuron to which theparticular input branch belongs remained inactive in response to theactivated set of inputs.
 7. The method of claim 1 wherein moderating thestability of connections between input branches and inputs includesincreasing the stability of a connection between a particular inputbranch and a particular input when the particular input was among theactivated set of inputs, the particular input branch was activated inresponse to the activated set of inputs, and the simulated neuron towhich the particular input branch belongs remained inactive in responseto the activated set of inputs.
 8. The method of claim 1 whereinmoderating the stability of connections between input branches andinputs includes increasing the stability of a connection between aparticular input branch and a particular input when the particular inputwas among the activated set of inputs, the particular input branch wasactivated in response to the activated set of inputs, and the simulatedneuron to which the particular input branch belongs was activated inresponse to the activated set of inputs.
 9. The method of claim 1wherein forming initial connections between various input branches andvarious inputs comprises forming initial connections between variousinput branches and various inputs based on the activated set of inputs.10. The method of claim 9 wherein forming initial connections betweenvarious input branches and various inputs comprises forming initialconnections between various input branches and various inputs based on apattern of activity generated in the input branches and the simulatedneurons in response to the activated set of inputs.
 11. The method ofclaim 1 wherein moderating the stability of connections between inputbranches and inputs comprises one or more of forming, increasing thestability of, decreasing the stability of, and severing connectionsbetween input branches and inputs based on the activated set of inputsand a pattern of activity generated in the input branches and simulatedneurons in response to the activated set of inputs.
 12. A method forgenerating a simulated neural circuit, the method comprising: generatinga first array of simulated neurons having output branches; generating asecond array of simulated neurons having input branches; activating aset of the output branches of the first array of simulated neurons; andgrowing one or more of the input branches of the simulated neurons ofthe second array of simulated neurons, wherein the growth of at leastone particular input branch is influenced in the direction of one of theactivated output branches of the first array of simulated neurons. 13.The method of claim 12 further comprising simulating the release of anattractant by one or more of the activated output branches in responseto activating the set of output branches, wherein: the particular inputbranch is configured to grow in the direction of the attractant; andgrowing the input branches of the simulated neurons of the second arrayof simulated neurons comprises growing the particular input branch inthe direction of the attractant.
 14. A computer program for moderatingconnections in a simulated neural circuit, wherein the simulated neuralcircuit includes a plurality of simulated neurons having input branchesthat are configured to connect to a plurality of inputs and activate inresponse to activity in the inputs to which the input branches areconnected, and wherein the simulated neurons are configured to activatein response to activity in their input branches, the computer programincluding instructions that, when executed, cause a computer to: forminitial connections between various input branches and various inputs;activate a set of the inputs; and moderate the stability of connectionsbetween input branches and inputs based on the activated set of inputsand a pattern of activity generated in the input branches and simulatedneurons in response to the activated set of inputs.
 15. The computerprogram of claim 14 wherein the instructions that, when executed, causea computer to moderate the stability of connections between inputbranches and inputs include instructions that, when executed, cause acomputer to decrease the stability of a connection between a particularinput branch and a particular input when the particular input was notamong the activated set of inputs, the particular input branch remainedinactive in response to the activated set of inputs, and the simulatedneuron to which the particular input branch belongs was activated inresponse to the activated set of inputs.
 16. The computer program ofclaim 14 wherein the instructions that, when executed, cause a computerto moderate the stability of connections between input branches andinputs include instructions that, when executed, cause a computer todecrease the stability of a connection between a particular input branchand a particular input when the particular input was not among theactivated set of inputs, the particular input branch was activated inresponse to the activated set of inputs, and the simulated neuron towhich the particular input branch belongs remained inactive in responseto the activated set of inputs.
 17. The computer program of claim 14wherein the instructions that, when executed, cause a computer tomoderate the stability of connections between input branches and inputsinclude instructions that, when executed, cause a computer to decreasethe stability of a connection between a particular input branch and aparticular input when the particular input was not among the activatedset of inputs, the particular input branch was activated in response tothe activated set of inputs, and the simulated neuron to which theparticular input branch belongs was activated in response to theactivated set of inputs.
 18. The computer program of claim 14 whereinthe instructions that, when executed, cause a computer to moderate thestability of connections between input branches and inputs includeinstructions that, when executed, cause a computer to decrease thestability of a connection between a particular input branch and aparticular input when the particular input was among the activated setof inputs, the particular input branch remained inactive in response tothe activated set of inputs, and the simulated neuron to which theparticular input branch belongs remained inactive in response to theactivated set of inputs.
 19. The computer program of claim 14 whereinthe instructions that, when executed, cause a computer to moderate thestability of connections between input branches and inputs includeinstructions that, when executed, cause a computer to decrease thestability of a connection between a particular input branch and aparticular input when the particular input was among the activated setof inputs, the particular input branch was activated in response to theactivated set of inputs, and the simulated neuron to which theparticular input branch belongs remained inactive in response to theactivated set of inputs.
 20. The computer program of claim 14 whereinthe instructions that, when executed, cause a computer to moderate thestability of connections between input branches and inputs includeinstructions that, when executed, cause a computer to increase thestability of a connection between a particular input branch and aparticular input when the particular input was among the activated setof inputs, the particular input branch was activated in response to theactivated set of inputs, and the simulated neuron to which theparticular input branch belongs remained inactive in response to theactivated set of inputs.
 21. The computer program of claim 14 whereinthe instructions that, when executed, cause a computer to moderate thestability of connections between input branches and inputs includeinstructions that, when executed, cause a computer to increase thestability of a connection between a particular input branch and aparticular input when the particular input was among the activated setof inputs, the particular input branch was activated in response to theactivated set of inputs, and the simulated neuron to which theparticular input branch belongs was activated in response to theactivated set of inputs.
 22. The computer program of claim 14 whereinthe instructions that, when executed, cause a computer to form initialconnections between various input branches and various inputs compriseinstructions that, when executed, cause a computer to form initialconnections between various input branches and various inputs based onthe activated set of inputs.
 23. The computer of claim 22 wherein theinstructions that, when executed, cause a computer to form initialconnections between various input branches and various inputs compriseinstructions that, when executed, cause a computer to form initialconnections between various input branches and various inputs based on apattern of activity generated in the input branches and the simulatedneurons in response to the activated set of inputs.
 24. The computerprogram of claim 14 wherein the instructions that, when executed, causea computer to moderate the stability of connections between inputbranches and inputs comprise instructions that, when executed, cause acomputer to perform one or more of forming, increasing the stability of,decreasing the stability of, and severing connections between inputbranches and inputs based on the activated set of inputs and a patternof activity generated in the input branches and simulated neurons inresponse to the activated set of inputs.
 25. A computer program forgenerating a simulated neural circuit, the computer program beingembodied on a tangible, computer-readable medium and includinginstructions that, when executed, cause a computer to: generate a firstarray of simulated neurons having output branches; generate a secondarray of simulated neurons having input branches; activate a set of theoutput branches of the first array of simulated neurons; and grow one ormore of the input branches of the simulated neurons of the second arrayof simulated neurons, wherein the growth of at least one particularinput branch is influenced in the direction of one of the activatedoutput branches of the first array of simulated neurons.
 26. Thecomputer program of claim 25 further comprising instructions that, whenexecuted, cause a computer to simulate the release of an attractant byone or more of the activated output branches in response to activatingthe set of output branches, wherein: the particular input branch isconfigured to grow in the direction of the attractant; and theinstructions that, when executed, cause a computer to grow the inputbranches of the simulated neurons of the second array of simulatedneurons comprise instructions that, when executed, cause a computer togrow the individual branch in the direction of the attractant.
 27. Asimulated neural circuit comprising: a plurality of inputs; and aplurality of simulated neurons having input branches that are connectedto one or more inputs and that activate in response to activity in theinputs to which the input branches are connected, wherein: the simulatedneurons are configured to activate in response to activity in theirinput branches, and the simulated neural circuit is configured tomoderate the stability of the connections between input branches andinputs based on patterns of activity generated in the input branches andsimulated neurons in response a pattern of activity in the inputs.
 28. Asystem for generating a simulated neural circuit, the system comprising:a processor configured to: generate a first array of simulated neuronshaving output branches; generate a second array of simulated neuronshaving input branches; activate a set of the output branches of thefirst array of simulated neurons; and grow one or more of the inputbranches of the simulated neurons of the second array of simulatedneurons, wherein the growth of at least one particular input branch isinfluenced in the direction of one of the activated output branches ofthe first array of simulated neurons.